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1. Introduction 

1.1. Pattern avoidance. Let n = (ni, . . . , 7r„) G & n and r 6 6 t be two per- 
mutations. An occurrence of r in tt is a subsequence tt' = (tt^, . . . , n ik ) such that 
1 ^ i\ < 12 < ■ ■ ■ < ik ^ n and tt' is order-isomorphic to r; in such a context r is usu- 
ally called a pattern. We say that tt avoids r, or is r- avoiding, if there is no occurrence 
of r in 7T. For example, let tt = 83176254, and n = 1234, r 2 = 1243, r 3 = 4213. Then 
it is easy to see that tt avoids Ti, contains exactly one occurrence of T2, namely, 1254, 
and contains six occurrences of T3, which are 8317, 8316, 8315, 8314, 8325, 8324. 

The set of all r- avoiding permutations in & n is denoted <5 n (r). For an arbitrary finite 
collection of patterns T, we say that tt avoids T if tt avoids any r G T; the corresponding 
subset of & n is denoted & n (T). By F T and F T we denote the corresponding (ordinary) 
generating functions F T (x) = E^o \&n(r)\x n and F T (x) = EZo \&n(T)\x n . 

The following simple symmetry arguments allow to decrease the variety of different T's 
to be considered. Define the reversal and the complementation operations as follows: 
r(7Ti,7r 2 , . . . ,7r„) = (7r n ,7r n _i,...,7Ti), c(vri,7r 2 , . . . , vr„) = (n + 1 -7Ti, n + 1 - 7r 2 , . . . , n + 
1 — 7r n ). It is easy to see that the following four statements are equivalent: 

(i) 7r avoids r; 

(ii) r{n) avoids r(r); 

(iii) c(ir) avoids c(r); 

(iv) 7T _1 avoids r _1 . 

More generally, denote by G the group of transformations generated by r, c, and the 
usual group inverse operation (it is easy to see that G is isomorphic to the dihedral 
group D%). The for any g G G one has Ft(x) = F g (r)(x). 

The first paper devoted entirely to the study of permutations avoiding certain patterns 
(restricted permutations) appeared in 1985 (see [SS]). Currently there exist more than 
fifty papers on this subject. 



1.2. Chebyshev polynomials of the second kind. Chebyshev polynomials of the 
second kind (in what follows just Chebyshev polynomials) are defined by 

tt f a\ sin ^ r + 1 ^ 
U r (cosO) = 



smv 

for r ^ 0. Evidently, U r (x) is a polynomial of degree r in x with integer coefficients. For 
example, U (x) = 1, Ui(x) = 2x, U 2 (x) = 4x 2 — 1, and in general, U r (x) = 2xU r - 1 (x) — 
U r -2(x). Chebyshev polynomials were invented for the needs of approximation theory, 
but are also widely used in various other branches of mathematics, including algebra, 
combinatorics, and number theory (see [Ri]). 
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For k > 1 we define Rk{x) by 

_ 2tU k -,{t) 1 

For example, -Ri(x) = 1, R2(x) = j^, and i?3(x) = jf^:- It is easy to see that for any 
k, Rk(x) is rational in x. These rational functions arise in some of the results below. 

1.3. Preliminaries. Apparently, for the first time the relation between restricted 
permutations and Chebyshev polynomials was discovered by Chow and West in [CW]. 
The main result of [CW] can be formulated as follows. 

Theorem 1.1. ([CW, Th. 3.1]) Let T x = {321, (2, 3, ... , k, 1)} ; T 2 = {132, (1,2,..., 
k)}, and T 3 = {132, (2, 3, ... , k, 1)}, then : 

(i) F Tl {x) = R k (x); 

(ii) F T2 (x) = R k (x); 

(iii) F T3 {x) = R k {x). 

The original proof is based on the use of transfer matrices (see Sec. 2 below). Several 
different proofs of various parts of this theorem appeared recently in [Kr, Th. 9] (part 
(i)), [MV1, Th. 3.1] and [Kr, Th. 2] (part (ii)), and [Kr, Th. 6] (part (iii)). They all are 
based on the use of continued fractions; the latter approach to restricted permutations 
was initiated in [RWZ] and developed in [MV1, JR, Kr]. In fact, there are two different 
ways to use continued fractions. One of them is more geometrical, and is based on the 
relation between continued fractions and Dyck paths discovered by Flajolet (see Sec. 3 
below). Another is more analytical, and is based on the study of block decompositions 
(see Sec. 4 below). 

In this paper we describe these three approaches, and present several ways to extend 
and generalize the above result. All our results deal with multiple (mainly, double) 
restrictions, of which one belongs to ©3 and others to Some of the results have been 
published previously, and we state them here without a proof. Observe that modulo 
standard symmetry operations (complement, reversal, and inversion), there are only 
two nonequivalent patterns in <5 3 ; we choose them to be 132 and 321. 

For the sake of brevity, we denote by [k] the identity pattern (1, 2, . . . , k) G & k , and by 
[k, m] the two-layered pattern (m + 1, m + 2, . . . , k, 1, 2, . . . , m) G &k- In general, we say 
that r G 6^ is a layered pattern if it can be represented as r = (r°, r 1 , . . . , r r ), where 
each of r l is a nonempty permutation of the form r l = (m i+ i + l,m i+ i + 2, . . . ,m,) 
with k = m > mi > ■ ■ • > m r > m r+ i = 0; in this case we denote r by [mo, . . . , m r ]. 
Observe that our definition slightly differs from the one used in [Bo, MV3]: their layered 
patterns are exactly the complements of our layered patterns. 

The authors are grateful to C. Krattenthaler and to an anonymous referee for valuable 
comments that helped us to improve the presentation. 

2. Transfer matrices 

The main idea behind the transfer matrix approach can be described as follows (see 
[Stl, Th. 4.7.2]). Consider a directed multigraph on n vertices v±, . . . ,v n , and let A 
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denote its weighted adjacency matrix, that IS ^ Qj'ij IS the number of edges directed from 
V{ to Vj. Then the generating function for the number of walks from v r to v s is given 
by 

(-iy+ s det(I-xA;r,s) 

det(I-xA) 1 ' } 

where / is the identity matrix and det(S; r, s) is the minor of B with the rth row and 
sth column deleted. 

To apply this approach, one has to construct a bijection between the permutations in 
question and walks in an appropriate directed graph. We describe below two bijections 
of this type: the first based on generating trees, and the second based on Dyck paths. 

2.1. Generating trees. Following [W], a generating tree is a rooted labeled tree with 
the property that if V\ and t>2 are any two nodes with the same label and I is any label, 
then V\ and t>2 have exactly the same number of children with the label /. To specify 
a generating tree it therefore suffices to specify: 

1. the label of the root, and 

2. a set of succession rules explaining how to derive from the label of a parent the 
labels of all of its children. 

Example 2.1. (The complete binary tree) Since all the nodes in the complete binary 
tree are similar, it is enough to use only one label, which we choose to be 2. So we get 
the following description: 

Root: (2) 

Rule: (2) -> (2)(2). 

Example 2.2. (The Fibonacci tree) Here we have nodes of two different types, so we 
use two labels: 1 for a non-breeding pair and 2 for a breeding pair. We thus get: 

Root: (1) 

Rules: (1) -> (2), (2) - (1)(2) 

Given a generating tree, one assigns to it a directed graph whose vertices correspond 
to labels and edges from Zj to lj correspond to the occurrences of lj in the succession 
rule (Zj) — > • • • . The graphs corresponding to the above two examples are shown in 
Figure 1. 




Figure 1. Directed graphs for the complete binary tree and the Fi- 
bonacci tree 



Given a permutation r, one defines a rooted tree as follows. The nodes on level n are 
precisely the elements of & n (r). The parent of a permutation 7r = (jri, 7T2, . . . , vr n ) is 
the unique permutation n' — {jii, . . . , 7Tj_i, 7r,-+i, . . . , n n ) such that ttj = n. We denote 
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the resulting tree T(r). Similarly, the tree corresponding to the set & n (T) is denoted 
by T(T). 

Chow and West [CW] proved that the succession rules for the tree T(123, (k— 1, . . . , 1, k)) 
are 

(Z)->(2)-..(Z)(Z + l), l<k-l 
(fc-l)->(2)...(fc-l)(fc-l), 
and the label of the root is (2). The corresponding graph is shown in Figure 2. 




Figure 2. The directed graph for T (123, (k - 1, ... ,2,1, k)) 



The corresponding transfer matrix is 

110 
1110 
1111 



1111 
1111 
1111 
1111 









1 

1 1 

1 1 1 

1 1 2 



Besides, Chow and West proved that the graphs, and hence the transfer matrices for 
T(213, (k, 1, 2 . . . , k — 1)) and T(213, (1, 2, ... , k)) are exactly the same (though the 
succession rules may vary). The number of permutations in & n (T) (in all cases) is 
thus equal to the number of walks of length n starting from the vertex 2. However, 
since each vertex is connected to vertex 2 by exactly one edge, this number is equal 
to the number of walks of length n + 1 starting at vertex 2 and ending at the same 
vertex. The generating function for this number is given by (2.1) with A = A k . It is 
proved in [CW] that the determinants in question satisfy linear recurrences of order 
two very similar to that for Chebyshev polynomials, which almost immediately yields 
Theorem 1.1, since T\ = c({123, (k - 1, . . . , 2, 1, k)j), T 2 = r o c({213, (1, 2, ... , k)}), 
and T 3 = r o c({213, (k, 1, 2, . . . , , k - 1)}). 

2.2. Dyck paths. A Dyck path is a path in the plane integer lattice Z 2 , consisting of 
up-steps (1,1) and down-steps (1,-1), which never passes below the 

Following [Kr], we define a bijection $ between permutations in (5„(132) and Dyck 
paths from the origin to the point (2n, 0). Let ix = (tti, . . . , 7r n ) be a 132- avoiding 
permutation. We read the permutation n from left to right and successively generate a 
Dyck path. When 7ij is read, then in the path we adjoint as many up-steps as necessary, 
followed by a down-step from height hj + 1 to height hj (measured from the x-axis), 
where hj is the number of elements in 7^+1,7^+2, ■ ■ ■ , vr n which are larger that nj. 
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For example, let it = 534261. The first element to be read is 5. There is one element 
in 34261 which is larger than 5, therefore the path starts with two up-steps followed 
by a down-step, thus reaching height 1. Next 3 is read. There are 2 elements in 4261 
which are larger than 3, therefore the path continues with two up-steps followed by a 
down-step, thus reaching height 2. Etc. 

Conversely, given a Dyck path starting at the origin and returning to the x-axis, the 
obvious inverse of the bijection <3> produces a 132-avoiding permutation. 

It is proved in [Kr] that the bijection <£> takes permutation in (5 n (132, [k]) to Dyck paths 
that never pass above the line y = k — 1. Evidently, such paths correspond bijectively 
to walks of length 2n starting at vertex in the graph shown in Figure 3. 




Figure 3. The directed graph for Dyck paths in a strip 



Using again (2.1) one gets Theorem l.l(ii). A further study of the bijection $ yields 
part (in) of the same Theorem. 

3. Continued fractions 

The relation between restricted permutations and continued fractions was discovered by 
Robertson, Wilf, and Zeilberger in [RWZ]. The main result in [RWZ] can be formulated 
as follows. Let G r T (x) be the generating function for the number of permutations in 
<5 n (132) containing a pattern r exactly r times. 

Theorem 3.1. (Robertson, Wilf, and Zeilberger [RWZ, Th. 1]) 

XZ^ 

xz^ 2 ' 



in which the jth numerator is xz( 2 ). 

To prove this, let it be a permutation avoiding 132. Then each letter in it to the left 
of n must be greater than any letter to the right of n. Thus, if ir — (it', n, it") (where 
both it' and it" must necessarily be 132-avoiding), then 

(123)tt = (123)7t' + (12)tt' + (123)tt", 

where (t)tt is the number of occurrences of r in it. It follows that the generating 
function 

F(x,y,z)= J2 ^ (1 N (12) ^ (123K 

7ree(132) 
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satisfies the equation F(x, y, z) — 1 + xF(xy, yz, z)F(x, y, z). Equivalently, 

F{x,y,z) = — -, 

1 — xr [xy, yz, z) 

and the theorem follows by induction after plugging in y = 1. 

This result was generalized by Mansour and Vainshtein [MV1], by Krattenthaler [Kr], 
and by Jani and Rieper [JR] to the case of permutations containing the pattern [k] = 
12 ... k exactly r times. It turns out that 

EW* r = ^ — > (3-i) 



r>0 i £f: 



xz d2 



xz d 3 



where d 3 = ( 3 k _\). 



The proof in [MV1] is a straightforward generalization of the above proof of Theorem 
3.1. The proof in [Kr] is based on the bijection <3> between 132-avoiding permutations 
and Dyck paths described in the previous section and on the result of Flajolet [Fl, 
Th. 1] presenting the generating function for the Dyck paths in terms of continued 
fractions. The proof of [JR] is based on a bijection between 132-avoiding permutations 
and rooted ordered trees, which can be obtained from the bijection <3> via the standard 
bijection between rooted ordered trees and Dyck paths through a depth-first traversal of 
the trees (see [St 2, Prop. 6.2.1, Cor. 6.2.3]). For further generalizations and interesting 
combinatorial applications see [BCS]. 

It was observed in [MV1] that Rk{x) is the kth. approximant for the continued fraction 

1 




1 - '•• 

so (3.1) for r = immediately gives Theorem l.l(ii). 

Paper [Kr] contains the description of a bijection \1/ between 123-avoiding permutations 
and Dyck paths. This bijection, combined with Roblet and Viennot's continued fraction 
representation of the generating function for Dyck paths [RV, Prop. 1] gives the first 
part of Theorem 1.1. 

4. Block decompositions 

The core of this approach initiated by Mansour and Vainshtein [MV2] lies in the study 
of the structure of 132-avoiding permutations, and permutations containing a given 
number of occurrences of 132. 

Let us start with the simplest case of 132-avoiding permutations. It was noticed in 
[MV2] that if a G ©„(132) and at = n, then a = (a', n, a") where a' is a permutation 
of the numbers n—t+1, n—t+2, . . . , n, a" is a permutation of the numbers 1, 2, ... , n—t, 
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and both a' and a" avoid 132. This representation is called the block decomposition 
of a, see Figure 4. 



n-t 



a' 



a' 



Figure 4. The block decomposition for a G 6„(132) 



The simple observation allows to formulate a general result concerning permutations 
avoiding 132 and arbitrary pattern r = (ti, . . . ,7%) G 6^(132). Recall that Tj is said 
to be a right-to-left maximum if Tj > Tj for any j > i. Let m = k,rrii, . . . ,m r be the 
right-to- left maxima of r written from left to right. Then r can be represented as 

r = (r°, m , r 1 , m x , . . . , r r , m r ), 

where each of t 1 may be possibly empty, and all the entries of r* are greater than all the 
entries of r* +1 . Define the ith prefix of r by n l = (r°, m , . . . , r l , m,) for 1 ^ i ^ r and 
7T° = r°, 7T _1 = 0. Besides, the ith sii/^r of r is defined by a 1 = (T l ,rrii, . . . ,T r ,m r ) 
for < i ^ r and a r+1 = 0. 

Theorem 4.1. ([MV2, Th. 1]) For any r G ©^(132), F T (x) is a rational function 
satisfying the relation 

r 

F T (x) = l + xJ2i F *i(x) ~ F ni -i(x))F ffj (x). 

3=0 

The proof is rather straightforward. Let a = (a',n,a") be the block decomposition 
of a G (5 n (132). It is easy to see that a contains r if and only if there exists i, 
^ % ^ r + 1, such that a' contains tt 1-1 and a" contains a 1 . Therefore, a avoids r if 
and only if there exists i, ^ i ^ r, such that a' avoids and contains tt* -1 , while a" 
avoids a 1 . We thus get the following relation: 

n r 

^H = EE^ 1 (t-i)^(^-t), 

t=l j=0 

where f T (n) = |S n (132,r)|, and f^{n) is the number of permutations in & n {r) con- 
taining p at least once. To obtain the recursion for F T (x) it remains to observe that 

for any I and j, and to pass to generating functions. Rationality of F T (x) follows easily 
by induction. 



RESTRICTED PERMUTATIONS AND CHEBYSHEV POLYNOMIALS 9 

Theorem 4.1 allows to reduce the calculation of F T (x) to finding similar functions for 
several simpler patterns. For example, if r = [k, m] is a two-layered pattern, then 
Theorem 4.1 gives 

F[ k , m ](x) = 1 + zF[fc_ m _i](z)F[ fe)m ](z) + x(F[ ktm ](x) - F [Jfc _ m _i](x))F[ m ](a;) (4.1) 

and hence F[fc jjn ](x) can be expressed via F[ m ](x) and F[fc_ m _i](x), (see Theorem 5.1 
below) . 

Consider now the case of permutations containing 132 exactly once. 



n-t+l 



n-t+2 



Figure 5. The block decomposition for a e 6„ containing 132 exactly once 



Theorem 4.2. Lei a e ©„ contain 132 exactly once. Then the block decomposition of 
a can have one of the following three forms: 

(i) there exists t such that a = (a',n,a"), where a' is a permutation of n — l,n — 
2, . . . , n — t + 1 containing 132 exactly once, and a" is a permutation of 1,2, ... , n — £ 
avoiding 132; 

(zzj £/iere exzs£s £ s«c/i £/ia£ a = (a',n,a"), where a' is a permutation of n — l,n — 
2, . . . , n — t + 1 avoiding 132, and a" is a permutation of 1,2, ... , n — £ containing 132 
exactly once; 

(Hi) there exist t,u such that a = (a',n — t + l,n, a",n — £ + 2, a'"), where a' is 
a permutation of n — l,n — 2, . . . ,n — £ + 3 avoiding 132, a" z's a permutation of 
n — t,n — t — 1, . . . ,n — u + 1 avoiding 132, and a'" is a permutation of 1,2, ... ,n — u 
avoiding 132. 

Proof. Let a G 6 n contain 132 exactly once. There are two possibilities: either the 
only occurrence of 132 in a does not contain n, or it contains n. In the first case we 
get immediately that any entry of a to the right of n is less than any entry of a to 
the left of n, since otherwise one gets an occurrence of 132 involving n. In the second 
case, let % n j be the occurrence of 132 in a. First, we have j — % + 1, since otherwise 
either (i + 1) n j or i n (i + 1) would be a second occurrence of 132 in a. Next, % 
immediately precedes n in a, since if / lies between i and n, then either / n j or i I j 
would be a second occurrence of 132 in a. Finally, any entry to the right of j is less 
than any entry between n and j, which in turn, is less than any entry to the left of % 
(the proof is similar to the analysis of the first case). □ 
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The block decompositions of types (i) and (ii) are similar to that for a G (5 n (132) (see 
Figure 4). The block decomposition of type (Hi) is shown in Figure 5. 

For a far reaching generalization of this idea, allowing to enumerate permutations with 
any given number of occurrences of 132, see [MV4]. 



5. Counting 132-restricted permutations 

Throughout this section we write F T (x) instead of i*i32 jT (x). 

5.1. Avoiding 132 and another pattern. By Theorem 1.1 (ii) and (iii), the pairs 
{132, [k]} and {132, [k, 1]} are Wilf- equivalent, that is F^(x) = F[ k;1 ](x). It turns out 
that this equivalence class can be extended as follows. 

Theorem 5.1. ([MV2, Th. 2.4]) For any m, 1 < m < k - I, 

F[k,m](x) = R k (x). 

This result follows immediately from Theorem 1.1 and (4.1). 



A further extension is provided by the following result. We say that r G (5^ is a wedge 
pattern if it can be represented as r = (t 1 , p 1 , . . . , r r , p r ) so that each of t 1 is nonempty, 
(p 1 , p 2 , . . . , p r ) is a layered permutation of 1, . . . , s for some s, and (t 1 , r 2 , . . . , r r ) = 
(s + 1, s + 2, . . . , k). For example, 645783912 is a wedge pattern; here r = 3, s = 5, 
r 1 = (6), r 2 = (7,8), r 3 = (9), p 1 = (4,5), p 2 = (3), p 3 = (1,2). Evidently, [k,m\ is a 
wedge pattern for any m. 

Theorem 5.2. ([MV2, Th. 2.6]) F T (x) = R k (x) for any wedge pattern r G 6 fe (132). 

This result follows easily from Theorem 4.1. The proof of the above two results are 
purely analytical. It would be very interesting to find a bijective proof of these results. 

The case of general layered patterns can be also expressed in terms of Chebyshev 
polynomials, using the same technique of block decompositions. Since the expressions 
become rather cumbersome, we present here only the case of a 3-layered pattern. 

Theorem 5.3. ([MV2, Th. 2.5]) For any k > m 1 > m 2 > 0, 

p , s _ ^ Ua+bWa+c-^U^t) + U b -!(t)U b (t) _ 1 

* [km] {x) ----- Ua+b{t)Ua+c{t)Ub+c[t) > 2v/i' 

where a = k — mi, b = mi — m 2 , c = m 2 . 

5.2. Avoiding 132 and containing another pattern. Denote by G r T (x) the gener- 
ating function for the number of permutations in (5 n (132) containing a pattern r G & k 
exactly r times. The continued fraction representation (3.1) for r = 1 immediately 
gives the following result. 

Theorem 5.4. ([MV1, Th. 3.1]) For any k > 1, 

G[ k ](x) = 772777, t 



u 2 (ty ' ~ 2^ 
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This result may be extended in two directions. First, let us fix r = 1 and consider 
other patterns r. The case of r = [k, 1] was studied in [Kr]. 

Theorem 5.5. ([Kr, Th. 7]) For k>2, 



4W k _ 2 (t)U k (ty 2^ 

The proof involves bijection $ and the result of Flajolet mentioned in Section 3. 

A more general case of r = [k,m] is investigated in [MV2], based on block decomposi- 
tions. 

Theorem 5.6. ([MV2, Th. 3.4]) For any k > m> 0, 

G ' k ^ {x) = 2tu k (t)u m (t)u k - m - 1 (ty 1 = v* 

Another direction would be to increase the value of r. A generalization of Theorem 5.4 
to the case 1 < r < k was obtained in [MV1], directly from (3.1). 

Theorem 5.7. ([MV1, Th. 3.1]) For anyr,l<r < k, 

u r k -\(t) , 1 



GUx) 



A similar generalization of Theorem 5.5 is as follows. 
Theorem 5.8. ([Kr, Th. 7]) For any r, 1 < r < k — 1, 

G - ( ^id^^m( 2 /) (2()I ^ f (fei) r/ ' " 

l\r 

The ideas behind the proof are the same as in the proof of Theorem 5.5. 
Theorem 5.7 can be extended to cover a wider range of r's. 
Theorem 5.9. ([MV1, Th. 4.1]) For anyr,l<r < k(k + 3)/2, 

wW &y-wi+\t) ^ v j J{ two 



2y^' 



where t = \/2^fx. 

The case of general r was treated in [Kr] . 
Theorem 5.10. ([Kr, Th. 3]) For any r, 

where the sum is over all nonnegative integers l±, ■ ■ ■ with 



and t = 1/2^/x. 
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5.3. Containing 132 exactly once and avoiding another pattern. Denote by 
H T (x) the generating function for the number of permutations in & n avoiding a pattern 
r G &k and containing 132 exactly once. We start from the following result obtained 
in [MV1]. 

Theorem 5.11. ([MV1, Th. 4.2]) For any k > 3, 

k-2 



The idea behind the proof is similar to that of the proof of Theorem 3.1 explained 
above in Section 3. 

This result can be extended to the case of general 2-layered patterns as follows. 
Theorem 5.12. (i) For any k > A, 



besides, 



(ii) For any k > 5, 



#[3,1] ( x ) 



x 3 



l-2x 



besides, 



\U JW U k _ 2 {t) J> ' 2^' 



_ :r 3 (l+:r) 

H[4 ' 2]{X) - (l-x)(l-3x + xiy 



(iii) For any k > 6 and any m, 3 < m < k/2, 

h *m{x) = wm(^ T=r 2 um + ejti 1 m) - 1 

+E^_i(t)E/ m _i(i)tf fc _ m _ 2 (t)^ , t = 

Remark. Clearly, the cases /c = 3, m = 2; A; = 4, 5, m = 3; k > 6, m > fc/2 are also 
covered, since H [k , m] (x) = H [Kk _ m] (x). 

Proof. By Theorem 4.2, we have exactly three possibilities for the block decomposition 
of an arbitrary a G & n . Let us write an equation for H[ k ,i](x) with k > 4. The contri- 
bution of the first decomposition above is xH[ k _ 2 ](x)F[k,i]( x ) + x (H{k,i]( x ) — H{k-2](%))- 
Here the first term corresponds to the case a' avoids [k — 2] and a" avoids [k, 1], while 
the second term corresponds to the case a' avoids [k, 1] but contains [k — 2], and a" = 0. 

The contribution of the second possible decomposition is xF[ k _ 2 ](x)H[ k! i](x); here a" 
contains 132, and hence is always distinct from 0. 
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Finally, the contribution of the third possible decomposition is 

x3 F[k-2]( x )F[k,i](x) + x 3 F [k _ 2] (x)(F [k;1] (x) - F [fc _ 2] (a;)). 

Here the first term corresponds to the case a', a" avoid [k — 2], a'" avoids [k, 1], while 
the second term corresponds to the case a' avoids [k — 2], a" avoids [k, 1] but contains 
[k - 2], and a'" = 0. 

Solving the obtained linear equation and using Theorems 1.1, 5.11, and well known 
identities involving Chebyshev polynomials (see e.g. [MV2, Lem. 4.1]), we get the 
desired expression for H[ k ,i\(x), k > 4. 

In the case k — 3, the contributions of the first and the second decompositions degen- 
erate to £#[3,1] (x) each, while the contribution of the third decomposition degenerates 
to a; 3 (which means that the only permutation having this decomposition is 132 itself). 
The result follows immediately. 

Let us consider now the case of H\ k2 Ax) with k > 5. The contribution of the first 
decomposition is xH[ k _ 3 j(x)F[ kj2 ](x) + x(H[ kj2 ](x) — H[ k _^(x)) F[ 2 ](x) . Here the first 
term corresponds to the case a' avoids [k — 3] and a" avoids [k,2], while the second 
term corresponds to the case a' avoids [k, 2] but contains [k — 3], and a" avoids [2]. 

The contribution of the second decomposition is xF[ k _ 3 j(x)H[ k2 \(x). 

Finally, the contribution of the third decomposition is 

x 3 F[ 2 fc _ 3] {x)F [K2] (x) + x 3 F [k „ 3] (x) (F [fc>2] (x) - F [fc _ 3 ] (x))F [2] (x) 

+x 3 (F [k „ 2] (x) - F [k _ 3] (x))F [2] (x). 

Here the first term corresponds to the case a', a" avoid [k — 3], a'" avoids [k,2], the 
second term corresponds to the case a' avoids [k — 3], a" avoids [k, 2] but contains 
[k — 3] , and a'" avoids [2] , while the third term corresponds to the case a' avoids [k — 2] 
but contains [k — 3], a" = 0, and a'" avoids [2]. 

The expression for H[ k ^(x) follows easily from this equation and Theorems 1.1 and 
5.11. 

The case k = 4 is treated similarly. 

For general m, 3 < m < k — m, the contribution of the first decomposition equals 
xH[ k - m -ii(x)F[ kiTn ](x) + x(H[k, m ](x) -if[ fc _ m _i](z))F[ m ](z), the contribution of the sec- 
ond decomposition equals xF[ k - m -i](x)H[ kjm ](x) + x(F[ ktm ](x) - F [fc _ m _i](x))if[ m ](x), 
and the contribution of the third structure decomposition 

£ 3 ^[L m _i](a;)^[fc,m](a;) + x 3 F [k _ m _ 1] (x)(F [ktm] (x) - F[ fc _ m _i](x))F[ m] (a;) 

+X 3 (F [kM (x) - ^[fc-m-l] (x)) F[ m _!] (x)F[ m j (x). 

The final result again follows from Theorems 1.1 and 5.11. □ 

5.4. Containing 132 and another pattern exactly once. Denote by 3> T (x) the 
generating function for the number of permutations in & n containing both 132 and a 
pattern r G & k exactly once. We start from the following result. 
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Theorem 5.13. For any k > 1, 



Proof. The three possible block decompositions of permutations containing 132 exactly 
once are described in Theorem 4.2. Let us find the recursion for It is easy to 

see that the contribution of the first decomposition equals 

x$[ fc _i](x)F[ fe] (x) + £#[ fc _i](z)G[ fe] (:r), 

the contribution of the second decomposition equals 

xG \k-l}( x ) H [k]( x ) + x F[k-l](x)® [k ](x), 

while the contribution of the third decomposition equals 

2x 3 G\ k _ 1] (x)F [k _ 1] (x)F [k] (x) + x'F^^G^x). 

Solving the obtained recursion with the initial condition $[ 2 ]( a; ) — and using Theo- 
rems 1.1, 5.4, and 5.11, we get the desired result. □ 

In particular, for k = 3 we get &[3](x) = 2x 5 (l — 2x)~ 3 , which means that the number of 
permutations in & n containing both 132 and 123 exactly once equals (n — 3)(n — 4)2 n ~ 5 
(see [R, Th. 1.3.18]). 

Similarly to the previous section, this result can be extended to the case of general 
2-layered patterns. Since the answers become very cumbersome, we present here only 
the simplest case. 

Theorem 5.14. For any k > 4, 

I 1 s^k-4 TT 2(,\ , Uk-a(t) | Uk(t) \ 

^ 2tUl_ 2 (t) 2-,i=l U i \ L > "T 2rt/ fc _ 1 (t) "T U k ^(t) J ' 

where t = l/2y/x. 

5.5. Generalizations. Here we present several directions to generalize the results 
of the previous sections. The first of these directions is to consider more than one 
additional restriction. For example, the following result is true. Let F TltT2 (x) be the 
generating function for the number of permutations in <5 n (132, t±, r 2 ). Assume that 
Ti = [k, to], k — m ^ to, and r 2 = [I]. It is easy to see that the only interesting case is 
I > k — to, since otherwise F[k, m ],[(\(x) = 

Theorem 5.15. Let I > k — m > m, then 

F[k,m],Vl( X ) = Rk ^ X ) ~ { x Rk-m( x )Rm(x)) l+m ~ (Rk( x ) ~ R k - m {x)). 

Proof. Let a G (5 n (132), then either a = (which means that n = 0), or a = 
(a',n,a"), where both a' and a" avoid 132. According to this dichotomy, we get the 
following recursion: 

F[ k , m ],[i](x) = l + zF[ fc _ m _i](x)F[ fc>m ] >w (a;) + x(F [kj7nU i_ 1] (x) — F[ fe _ m _ 1 ](x))F[ m ](x). 
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Solving this recursion with the initial condition F^ m ^ k _ m \(x) = i 7 ^-™] (x) and using 
Theorem 1.1, we get the desired result. □ 

Similarly, let G Tl - T2 (x) be the generating function for the number of permutations in 
<5 n (132, Ti) that contain r 2 exactly once. As before, we assume T\ — [k,m], k — m ^ m, 
and r 2 = [/]. Once again, the case m ^ I is of no interest, since in this case G[ k m]-[i](x) = 
G {l] (x). 

Theorem 5.16. (i) Let I > k — m > m, then 

r, , rn M - 1 ( u m -i(t) \ l - m ( u k - m -,{t) \ l+m ' k 

^[k,m];[l]\ X J ~ U m (t)U k _ m (t) \ U m (t) ) \ U k _ m (t) ) 

m+l—j 



v fc-m Uj-m-2(t) ( U m -i{t) \ 3 1 1 

x \l^j=m+2 Uj-2(t)Uj-i(t) \ U m (t) J ^ L 



where t = l/2y/x. 

(ii) Let k — m > I > m, then 

G[ k , m ];[l](x) = 

1 ( Urn^M Uj-m-ljt) ( U m -! (t ) \ m ~ j 

Ul{t)Um{t) \ Um{t) ) \2^j=m+l Uj-l(t)Uj(t) \ Um(t) J " t " 



where t = 1/2^/x. 

Another possible direction is to replace 132 by some restriction of length 4 or more 
having a similar restrictive power. Define L p as the set of all patterns in & p of the 
form 7r 1 l7r 2 27r3, where ir 2 is nonempty. Evidently, L 3 = {132}; for p = 4 we get 
L 4 = {1324, 1423, 1342, 1432, 3142, 4132}, and so on. It turns out that L p is, in a sense, 
an analog of 132 for p > 3. For example, the following result is an analog of Theorem 
1.1 (ii) for p — 4. 

Theorem 5.17. For any k > 2, 

F{L 4 ,[k]}(x) = I + x + x 2 R k (x)R k _ 1 (x)(R k ^ 1 (x) + R k ^ 2 (x)). 

Proof. The main ingredient of the proof is the following description of the block de- 
compositions of permutations in <5 n (L 4 ). Let a G S n (L 4 ), then there exist ^ r < 
s < n — 1 such that either a — a±, n — 1, a 2 , n, as, or a = a±, n, a 2 , n — 1, 0:3, where 
a± is a permutation of the numbers s + 1, s + 2, . . . , n — 2, a 2 is a permutation of the 
numbers r + 1, r + 2, . . . , s, and 0:3 is a permutation of the numbers 1, 2, . . . , r. □ 

Consider now the case of a general p > 3. For an arbitrary n G & p we define a 
sequence a(n) of zeros and ones of length p — 1 as follows. First of all, we put 
ai(ji) = 1 if 7T p _i < 7T P and aiiji) = otherwise. If ai (ir) , . . . , aj (7r) are already 
determined, we put a J+ i(7r) = 1 if the length of the maximal increasing subsequence in 
(n k ^j^i, iT k _j, . . . , n k ) is greater than that of (rrk-j, ■ ■ ■ , ^k), and aj + i(n) = otherwise. 
For example, let n = 7346215, then a^n) = 1, a 2 {it) = 0, 03(71") = 0, a 4 (7r) = 0, 
a 5 (n) = 1, a 6 (7r) = 0. For an arbitrary sequence a = (a 1: . . . , a p _i) G Q p ~ l = {0, 1} P_1 
we denote by N(a) the number of permutations tt G 6 P such that a(7r) = a. The 
following result is a further generalization of Theorem 5.17. 
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Theorem 5.18. For any k > p — 2 and any p > 3, 

p— 3 p— 3 

F {Lpm (x) = J2^ l + ^ 2 Rk^)R^i(x) N(a)J[R k . j . a .(x). 

i=0 aeQP- 3 j=l 

As a generalization of Theorem 5.1 we get the following result. 
Theorem 5.19. (i) For any m, 1 < m < k — 2, 

P f („\ _ 1 j 1_ I Uk-2(t)Um-l(t) 

f{L 4 ,{k,m]}\x) — 1 -t- U 2 T tUk(t)U m (t) 

I Uk-m-2(t) I Uk-m-2(t) _|_ Uk-m-s(t) \ J. 1 

" r ' 2tU k (t)U m (t) \U k - m ^(t) ^ U k _ m _ 2 (t) J ' _ 2^- 

(ii) 

^{L 4 ,[fe,fe-i]}(^) = ^{L 4 ,[fc]}(a;)- 

6. Counting 321-restricted permutations 

The case of 321-restricted permutations is studied much less than the previous one. 
We start from the following generalization of Theorem l.l(i). 

Theorem 6.1. ([MV3, Th. 1.2(h)]) For k > 2 and any m, 1 < m < k - 1, 

F{321,[k,m]}( X ) = R k( x )- 

Therefore, the Wilf class of {132, [A;, to]} contains the pair {321, [/c,m]} as well. Once 
again, we have only an analytical proof of this result. Moreover, the ideas behind 
this proof are very different from these behind the proofs in the previous section. Let 
T = {321, [k,m]}; we define 

r+m 



A(n,r) = j>l)< r + 7 % )Mn-i), 
i=0 \ 1 / 



where fr(n) = \S n (T)\, and prove that A(n,r) satisfy a linear recurrence (see [MV3, 
Th. 2.3]). It follows that 

£(-*)'(* 7 ') ( Ft{x) ~ lfW)=o, 

i=o ^ ' \ j=o / 

where Cj is the j'th Catalan number. Using classical identities involving Catalan num- 
bers we get the desired result. 

In view of Theorems 5.1 and 5.2, it is a challenge to find a bijective proof of Theorem 
6.1. 

A striking analog of Theorem 5.7 is given by the following result. 
Theorem 6.2. ([Kr, Th. 10]) For k > 3 and anyr,l<r< k, 
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The proof of this result is based on the bijection ^ between 123-avoiding permutations 
with exactly r occurrences of the pattern [k, 1] and Dyck paths which start at the 
origin, return to the a>axis, and have exactly r peaks at height k. 

The case of a general 2-layered pattern remains intractable. Our computational exper- 
iments suggest that G r 32 i-[k i]( x ) = ^321- [k 2]( x ) for 1 ^ r ^ /c; however, we are unable 
to prove this. 
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